Commit Graph

402 Commits

Author SHA1 Message Date
Christian Heimes 994f90c077
bpo-45723: Fix detection of epoll (#30449) 2022-01-07 09:15:20 +01:00
Mark Shannon 342b93f9f2
bpo-46072: Add --with-pystats configure option to simplify gathering of VM stats (GH-30116)
* Simplify specialization stats collection macros.

* Add --enable-pystats option to configure.

* Update specialization summary script to handle larger number of kinds
2021-12-15 15:32:32 +00:00
Christian Heimes 91b59a3fcd
bpo-45847: Revert Port _ctypes partly to PY_STDLIB_MOD (GH-29747) (GH-29969) 2021-12-07 20:56:41 +01:00
Christian Heimes 4045392e0e
bpo-45847: Port _ctypes partly to PY_STDLIB_MOD (GH-29747)
Co-authored-by: Erlend Egeberg Aasland <erlend.aasland@innova.no>
2021-12-04 11:21:43 +01:00
Erlend Egeberg Aasland f7a62f2405
bpo-45847: Make socket module conditional (GH-29769) 2021-11-29 14:57:36 +01:00
Christian Heimes 4ebde73b8e
bpo-40280: Move hard-coded feature checks to configure (GH-29789)
Co-authored-by: Erlend Egeberg Aasland <erlend.aasland@innova.no>
2021-11-26 19:36:48 +01:00
Christian Heimes 8caceb7a47
bpo-40280: Add configure check for socket shutdown (GH-29795) 2021-11-26 15:16:54 +01:00
Christian Heimes 8af6481f6b
bpo-45847: Port _uuid to PY_STDLIB_MOD (GH-29741) 2021-11-24 10:20:37 +01:00
Erlend Egeberg Aasland 324527012f
bpo-45847: Port _posixshmem to PY_STDLIB_MOD (GH-29738) 2021-11-24 08:19:17 +01:00
Christian Heimes 9cf5646bb4
bpo-45847: Port _gdbm to PY_STDLIB_MOD (GH-29720) 2021-11-23 22:58:38 +01:00
Christian Heimes c6dec7e27a
bpo-45847: Port nis module to PY_STDLIB_MOD (GH-29699) 2021-11-22 15:18:41 +01:00
Christian Heimes d9cedabeba
bpo-45847: Port compression libs to PY_STDLIB_MOD (GH-29702) 2021-11-22 14:52:29 +01:00
Erlend Egeberg Aasland 5b946cadaa
bpo-45847: Port fcntl to Py_STDLIB_MOD (GH-29696)
Co-authored-by: Christian Heimes <christian@python.org>
2021-11-22 14:02:27 +01:00
Christian Heimes f201d261cf
bpo-45847: Port grp, spwd, termios, resource, syslog to PY_STDLIB_MOD (GH-29668) 2021-11-21 10:45:31 +01:00
Erlend Egeberg Aasland 6d430ef5ab
bpo-45774: Fix SQLite load extension autodetection (GH-29659) 2021-11-20 15:02:52 +01:00
Erlend Egeberg Aasland 29e5874d5a
bpo-45774: Autoconfiscate SQLite detection (GH-29507)
Co-authored-by: Christian Heimes <christian@python.org>
2021-11-19 15:10:41 +01:00
Christian Heimes 5275e59c0c
bpo-45573: check for ossaudiodev in configure (GH-29614) 2021-11-18 10:56:26 +01:00
Christian Heimes 0a9f69539b
bpo-45747: Detect gdbm/dbm dependencies in configure (GH-29467)
Co-authored-by: Erlend Egeberg Aasland <erlend.aasland@innova.no>
2021-11-10 20:26:55 +01:00
Christian Heimes 5b7c7cb104
bpo-45763: Detect compression build deps in configure (GH-29483) 2021-11-09 16:42:53 +01:00
Christian Heimes cbab997efb
bpo-45723: Prepare support for autoconf 2.71 (GH-29441) 2021-11-08 19:31:14 +01:00
Christian Heimes 57c50c9c7c
bpo-45723: Add helper macros and more caching to configure.ac (GH-29429)
Almost all checks are now cached by AC_CACHE_CHECK().

Common patterns are replaced by helper macros.

Variable names now use naming scheme ``ac_cv_func_$funcname``,
``ac_cv_lib_$library_$funcname``, or ``ac_cv_header_$headername_h``.

``SYS_SELECT_WITH_SYS_TIME`` is no longer used.

``uuid_create`` and ``uuid_enc_be`` are provided by libc on BSD. It is
safe to use ``AC_CHECK_FUNCS`` here.

Caching speeds up ./configure -C from ~ 4s to 2.6s on my system.

Co-authored-by: Erlend Egeberg Aasland <erlend.aasland@innova.no>
2021-11-08 08:06:31 +01:00
Christian Heimes e9594f6747
bpo-45731: Handle --enable-loadable-sqlite-extensions in configure (GH-29434) 2021-11-06 10:30:37 +01:00
Christian Heimes 9b0f45c069
bpo-43158: Regenerate configure again (GH-29433) 2021-11-05 21:51:26 +01:00
Christian Heimes 77a1f8d94c
bpo-43158: Use configure values for building _uuid extension (GH-29353) 2021-11-02 12:25:13 +01:00
Christian Heimes 77e3f224d6
bpo-45548: Remove checks for finite and gamma (GH-29206) 2021-10-25 18:25:43 +02:00
Christian Heimes 9942f42a93
bpo-45522: Allow to disable freelists on build time (GH-29056)
Freelists for object structs can now be disabled. A new ``configure``
option ``--without-freelists`` can be used to disable all freelists
except empty tuple singleton. Internal Py*_MAXFREELIST macros can now
be defined as 0 without causing compiler warnings and segfaults.

Signed-off-by: Christian Heimes <christian@python.org>
2021-10-21 06:12:20 -07:00
Victor Stinner 00ffc4513d
bpo-45440: Remove pymath.c fallbacks (GH-28977)
Remove fallbacks for missing round(), copysign() and hypot() in
Python/pymath.c. Python now requires these functions to build.

These fallbacks were needed on Visual Studio 2012 and older. They are
no longer needed since Visual Stuido 2013. Python is now built with
Visual Studio 2017 or newer since Python 3.6.
2021-10-15 19:45:34 +02:00
Victor Stinner 194a9526d8
bpo-45440: Require math.h isinf() to build (GH-28894)
Building Python now requires a C99 <math.h> header file providing
isinf(), isnan() and isfinite() functions.

Remove the Py_FORCE_DOUBLE() macro. It was used by the
Py_IS_INFINITY() macro.

Changes:

* Remove Py_IS_NAN(), Py_IS_INFINITY() and Py_IS_FINITE()
  in PC/pyconfig.h.
* Remove the _Py_force_double() function.
* configure no longer checks if math.h defines isinf(), isnan() and
  isfinite().
2021-10-13 23:27:50 +02:00
Inada Naoki ad970e8623
bpo-29410: Change the default hash algorithm to SipHash13. (GH-28752)
Co-authored-by: Erlend Egeberg Aasland <erlend.aasland@innova.no>
Co-authored-by: Christian Heimes <christian@python.org>
2021-10-10 17:29:46 +09:00
Victor Stinner 1ee0f94d16
bpo-41710: PyThread_acquire_lock_timed() uses sem_clockwait() (GH-28662)
On Unix, if the sem_clockwait() function is available in the C
library (glibc 2.30 and newer), the threading.Lock.acquire() method
now uses the monotonic clock (time.CLOCK_MONOTONIC) for the timeout,
rather than using the system clock (time.CLOCK_REALTIME), to not be
affected by system clock changes.

configure now checks if the sem_clockwait() function is available.
2021-10-01 09:55:28 +02:00
Victor Stinner 7834ff26cb
bpo-21302: Add nanosleep() implementation for time.sleep() in Unix (GH-28545)
Co-authored-by: Livius <egyszeregy@freemail.hu>
2021-09-25 14:36:26 +02:00
Livius 85a4748118
bpo-21302: Add clock_nanosleep() implementation for time.sleep() (GH-28111)
In Unix operating systems, time.sleep() now uses the clock_nanosleep() function,
if available, which allows to sleep for an interval specified with nanosecond precision.

Co-authored-by: Victor Stinner <vstinner@python.org>
2021-09-13 14:37:38 +02:00
Łukasz Langa be9de8721d
bpo-34602: Quadruple stack size on macOS when compiling with UBSAN (GH-27309) 2021-09-03 09:32:19 +02:00
Jakub Kulík 9032cf5cb1
bpo-43667: Fix broken Unicode encoding in non-UTF locales on Solaris (GH-25096) 2021-04-30 15:21:42 +02:00
Christian Heimes 39258d3595
bpo-43669: PEP 644: Require OpenSSL 1.1.1 or newer (GH-23014)
- Remove HAVE_X509_VERIFY_PARAM_SET1_HOST check
- Update hashopenssl to require OpenSSL 1.1.1
- multissltests only OpenSSL > 1.1.0
- ALPN is always supported
- SNI is always supported
- Remove deprecated NPN code. Python wrappers are no-op.
- ECDH is always supported
- Remove OPENSSL_VERSION_1_1 macro
- Remove locking callbacks
- Drop PY_OPENSSL_1_1_API macro
- Drop HAVE_SSL_CTX_CLEAR_OPTIONS macro
- SSL_CTRL_GET_MAX_PROTO_VERSION is always defined now
- security level is always available now
- get_num_tickets is available with TLS 1.3
- X509_V_ERR MISMATCH is always available now
- Always set SSL_MODE_RELEASE_BUFFERS
- X509_V_FLAG_TRUSTED_FIRST is always available
- get_ciphers is always supported
- SSL_CTX_set_keylog_callback is always available
- Update Modules/Setup with static link example
- Mention PEP in whatsnew
- Drop 1.0.2 and 1.1.0 from GHA tests
2021-04-17 11:36:35 +02:00
Jessica Clarke dec0757549
bpo-43179: Generalise alignment for optimised string routines (GH-24624)
* Remove m68k-specific hack from ascii_decode

On m68k, alignments of primitives is more relaxed, with 4-byte and
8-byte types only requiring 2-byte alignment, thus using sizeof(size_t)
does not work. Instead, use the portable alternative.

Note that this is a minimal fix that only relaxes the assertion and the
condition for when to use the optimised version remains overly strict.
Such issues will be fixed tree-wide in the next commit.

NB: In C11 we could use _Alignof(size_t) instead, but for compatibility
we use autoconf.

* Optimise string routines for architectures with non-natural alignment

C only requires that sizeof(x) is a multiple of alignof(x), not that the
two are equal. Thus anywhere where we optimise based on alignment we
should be using alignof(x) not sizeof(x).

This is more annoying than it would be in C11 where we could just use
_Alignof(x) (and alignof(x) in C++11), but since we still require only
C99 we must plumb the information all the way from autoconf through the
various typedefs and defines.
2021-03-31 12:12:39 +02:00
Roland Hieber e1f7769513
bpo-13501: allow choosing between readline and libedit (GH-24189)
In contrast to macOS, libedit is available as its own include file and
library on Linux systems to prevent file name clashes. So if both
libraries are available on the system, readline is currently chosen by
default; and if only libedit is available, it is not found at all. This
patch adds a way to link against libedit by adding the following
arguments to configure:

  --with-readline           link against libreadline (the default)
  --with-readline=editline  link against libeditline
  --with-readline=no        disable building the readline module
  --without-readline        (same)

The runtime detection of libedit vs. readline was already done in commit
7105319ada (2019-12-04, serge-sans-paille: "bpo-38634: Allow
non-apple build to cope with libedit (GH-16986)").

Fixes: GH-12076 ("bpo-13501 Build or disable readline with Editline")
Fixes: bpo-13501 ("Make libedit support more generic; port readline / libedit to FreeBSD")
Co-authored-by: Enji Cooper (ngie-eign)
Co-authored-by: Martin Panter (vadmium)
Co-authored-by: Robert Marshall (kellinm)
2021-02-08 17:05:25 -08:00
Victor Stinner 52a327c1cb
bpo-39465: Add pycore_atomic_funcs.h header (GH-20766)
Add pycore_atomic_funcs.h internal header file: similar to
pycore_atomic.h but don't require to declare variables as atomic.

Add _Py_atomic_size_get() and _Py_atomic_size_set() functions.
2020-12-23 03:41:08 +01:00
Pablo Galindo a57b3d30f6
bpo-41625: Expose the splice() system call in the os module (GH-21947) 2020-11-17 00:00:38 +00:00
Christian Heimes cd9fed6afb
bpo-41001: Add os.eventfd() (#20930)
Co-authored-by: Kyle Stanley <aeros167@gmail.com>
2020-11-13 19:48:52 +01:00
Ronald Oussoren 41761933c1
bpo-41100: Support macOS 11 and Apple Silicon (GH-22855)
Co-authored-by:  Lawrence D’Anna <lawrence_danna@apple.com>

* Add support for macOS 11 and Apple Silicon (aka arm64)
   
  As a side effect of this work use the system copy of libffi on macOS, and remove the vendored copy

* Support building on recent versions of macOS while deploying to older versions

  This allows building installers on macOS 11 while still supporting macOS 10.9.
2020-11-08 10:05:27 +01:00
Alexey Izbyshev 976da903a7
bpo-35823: subprocess: Use vfork() instead of fork() on Linux when safe (GH-11671)
* bpo-35823: subprocess: Use vfork() instead of fork() on Linux when safe

When used to run a new executable image, fork() is not a good choice
for process creation, especially if the parent has a large working set:
fork() needs to copy page tables, which is slow, and may fail on systems
where overcommit is disabled, despite that the child is not going to
touch most of its address space.

Currently, subprocess is capable of using posix_spawn() instead, which
normally provides much better performance. However, posix_spawn() does not
support many of child setup operations exposed by subprocess.Popen().
Most notably, it's not possible to express `close_fds=True`, which
happens to be the default, via posix_spawn(). As a result, most users
can't benefit from faster process creation, at least not without
changing their code.

However, Linux provides vfork() system call, which creates a new process
without copying the address space of the parent, and which is actually
used by C libraries to efficiently implement posix_spawn(). Due to sharing
of the address space and even the stack with the parent, extreme care
is required to use vfork(). At least the following restrictions must hold:

* No signal handlers must execute in the child process. Otherwise, they
  might clobber memory shared with the parent, potentially confusing it.

* Any library function called after vfork() in the child must be
  async-signal-safe (as for fork()), but it must also not interact with any
  library state in a way that might break due to address space sharing
  and/or lack of any preparations performed by libraries on normal fork().
  POSIX.1 permits to call only execve() and _exit(), and later revisions
  remove vfork() specification entirely. In practice, however, almost all
  operations needed by subprocess.Popen() can be safely implemented on
  Linux.

* Due to sharing of the stack with the parent, the child must be careful
  not to clobber local variables that are alive across vfork() call.
  Compilers are normally aware of this and take extra care with vfork()
  (and setjmp(), which has a similar problem).

* In case the parent is privileged, special attention must be paid to vfork()
  use, because sharing an address space across different privilege domains
  is insecure[1].

This patch adds support for using vfork() instead of fork() on Linux
when it's possible to do safely given the above. In particular:

* vfork() is not used if credential switch is requested. The reverse case
  (simple subprocess.Popen() but another application thread switches
  credentials concurrently) is not possible for pure-Python apps because
  subprocess.Popen() and functions like os.setuid() are mutually excluded
  via GIL. We might also consider to add a way to opt-out of vfork() (and
  posix_spawn() on platforms where it might be implemented via vfork()) in
  a future PR.

* vfork() is not used if `preexec_fn != None`.

With this change, subprocess will still use posix_spawn() if possible, but
will fallback to vfork() on Linux in most cases, and, failing that,
to fork().

[1] https://ewontfix.com/7

Co-authored-by: Gregory P. Smith [Google LLC] <gps@google.com>
2020-10-23 17:47:01 -07:00
Kyle Evans 1800c60080
bpo-40423: Optimization: use close_range(2) if available (GH-22651)
close_range(2) should be preferred at all times if it's available, otherwise we'll use closefrom(2) if available with a fallback to fdwalk(3) or plain old loop over fd range in order of most efficient to least.

[note that this version does check for ENOSYS, but currently ignores all other errors]

Automerge-Triggered-By: @pablogsal
2020-10-11 13:18:53 -07:00
wasiher 5798f78777
bpo-41340: Removed fallback implementation for strdup (GH-21634) 2020-07-27 12:28:45 +09:00
Christian Heimes 9b60e55db2
bpo-40637: Add option to disable builtin hashes (GH-20121)
Signed-off-by: Christian Heimes <christian@python.org>

Automerge-Triggered-By: @tiran
2020-05-15 14:54:53 -07:00
Victor Stinner c5fa364f4e
bpo-40514: Add --with-experimental-isolated-subinterpreters (GH-19926)
Add --with-experimental-isolated-subinterpreters build option to
configure: better isolate subinterpreters, experimental build mode.

When used, force the usage of the libc malloc() memory allocator,
since pymalloc relies on the unique global interpreter lock (GIL).
2020-05-05 16:41:11 +02:00
karl ding 360371f79c
bpo-40291: Add support for CAN_J1939 sockets (GH-19538)
Add support for CAN_J1939 sockets that wrap SAE J1939 protocol
functionality provided by Linux 5.4+ kernels.
2020-04-29 15:31:19 -07:00
Zackery Spytz 97e0de04b8
bpo-25780: Expose CAN_RAW_JOIN_FILTERS in the socket module (GH-19190)
Co-Authored-By: Stefan Tatschner <stefan@rumpelsepp.org>
2020-04-09 13:03:49 +01:00
Stefan Krah 815280eb16
bpo-39794: Add --without-decimal-contextvar (#18702) 2020-02-29 19:43:42 +01:00
Victor Stinner b8d1262e8a
bpo-39395: putenv() and unsetenv() always available (GH-18135)
The os.putenv() and os.unsetenv() functions are now always available.

On non-Windows platforms, Python now requires setenv() and unsetenv()
functions to build.

Remove putenv_dict from posixmodule.c: it's not longer needed.
2020-01-24 14:05:48 +01:00