Commit Graph

626 Commits

Author SHA1 Message Date
Christian Heimes c7f7069e77
bpo-34271: Add ssl debugging helpers (GH-10031)
The ssl module now can dump key material to a keylog file and trace TLS
protocol messages with a tracing callback. The default and stdlib
contexts also support SSLKEYLOGFILE env var.

The msg_callback and related enums are private members. The feature
is designed for internal debugging and not for end users.

Signed-off-by: Christian Heimes <christian@python.org>
2019-05-31 11:44:05 +02:00
Charles Pigott ad0daf5b37 bpo-31525: Increase minimum sqlite version number check (GH-12923) 2019-04-26 18:38:12 +03:00
Victor Stinner 574913479f
bpo-36635, bpo-36696: Fix setup.py on AIX (GH-12922)
xlc compiler doesn't support "-D define" flag only "-Ddefine".
2019-04-23 12:26:33 +02:00
Victor Stinner 23bace26ec
bpo-36635: Add _testinternalcapi module (GH-12841)
Add a new _testinternalcapi module to test the internal C API.

Move _Py_GetConfigsAsDict() function to the internal C API:
_testembed now uses _testinternalcapi to access the function.
2019-04-18 11:37:26 +02:00
Victor Stinner 5c75f37d47
bpo-36635: Change pyport.h for Py_BUILD_CORE_MODULE define (GH-12853)
Change PyAPI_FUNC(type), PyAPI_DATA(type) and PyMODINIT_FUNC macros
of pyport.h when Py_BUILD_CORE_MODULE is defined.

The Py_BUILD_CORE_MODULE define must be now be used to build a C
extension as a dynamic library accessing Python internals: export the
PyInit_xxx() function in DLL exports on Windows.

Changes:

* Py_BUILD_CORE_BUILTIN and Py_BUILD_CORE_MODULE now imply
  Py_BUILD_CORE directy in pyport.h.
* ceval.c compilation now fails with an error if Py_BUILD_CORE is not
  defined, just to ensure that Python is build with the correct
  defines.
* setup.py now compiles _pickle.c with Py_BUILD_CORE_MODULE define.
* setup.py compiles _json.c with Py_BUILD_CORE_MODULE define, rather
  than Py_BUILD_CORE_BUILTIN define
* PCbuild/pythoncore.vcxproj: Add Py_BUILD_CORE_BUILTIN define.
2019-04-17 23:02:26 +02:00
pxinwr 236d0b75c4 bpo-31904: Don't build the _crypt extension on VxWorks (GH-12833) 2019-04-15 11:02:20 +02:00
Christian Heimes 8abc3f4f91 bpo-36577: setup.py reports missing OpenSSL again (GH-12746)
[bpo-36146](https://bugs.python.org/issue36146) introduced another regression. In case of missing OpenSSL
libraries or headers, setup.py no longer reported _hashlib and _ssl to
be missing.

Signed-off-by: Christian Heimes <christian@python.org>



https://bugs.python.org/issue36577
2019-04-09 09:40:12 -07:00
xdegaye 2ee077f795 bpo-36544 : Fix regression introduced in bpo-36146 (GH-12708)
https://bugs.python.org/issue36544
2019-04-09 08:20:08 -07:00
Michael Felt b7eec94c0e bpo-36503: remove references to 'aix3' and 'aix4' (GH-12658) 2019-04-08 09:51:33 +09:00
Eric Snow c11183cdcf
bpo-36097: Use only public C-API in the_xxsubinterpreters module (adding as necessary). (gh-12359) 2019-03-15 16:35:46 -06:00
Victor Stinner 4d61e6e3b8
Revert: bpo-33608: Factor out a private, per-interpreter _Py_AddPendingCall(). (GH-11617) (GH-12159)
* Revert "bpo-36097: Use only public C-API in the_xxsubinterpreters module (adding as necessary). (#12003)"

This reverts commit bcfa450f21.

* Revert "bpo-33608: Simplify ceval's DISPATCH by hoisting eval_breaker ahead of time. (gh-12062)"

This reverts commit bda918bf65.

* Revert "bpo-33608: Use _Py_AddPendingCall() in _PyCrossInterpreterData_Release(). (gh-12024)"

This reverts commit b05b711a2c.

* Revert "bpo-33608: Factor out a private, per-interpreter _Py_AddPendingCall(). (GH-11617)"

This reverts commit ef4ac967e2.
2019-03-04 14:21:28 +01:00
Eric Snow bcfa450f21
bpo-36097: Use only public C-API in the_xxsubinterpreters module (adding as necessary). (#12003) 2019-03-01 16:50:31 -07:00
Victor Stinner cfe172dc6b
bpo-36146: Add TEST_EXTENSIONS to setup.py (GH-12129)
Add TEST_EXTENSIONS constant to setup.py to allow to not build test
extensions like _testcapi.

Changes:

* Add add_ldflags_cppflags() subfunction
* Rename add_compiler_directories() to configure_compiler().
* Remove unused COMPILED_WITH_PYDEBUG constant.
* Use self.add() rather than accessing directly self.extensions.
* Remove module_enabled() function: check differently if curses
  extension is built or not.
2019-03-01 18:21:49 +01:00
Victor Stinner c991f2415d
bpo-36146: Don't run code at setup.py top level (GH-12127)
* Move set_compiler_flags() calls and concurrent.future hack from
  module top-level to main()
* Remove unused variables 'macros' and 'libraries' from
  detect_multiprocessing().
* Move SUMMARY and CLASSIFIERS constants at the top, move
  set_compiler_flags() function below these constants.
* Add some empty new lines to respect PEP 8.
2019-03-01 17:19:04 +01:00
Victor Stinner 5ec33a1c25
bpo-36146: Split setup.py into subfunctions (GH-12125)
* Split PyBuildExt.detect_modules() huge function into subfunctions.
* Move curses, hashlib and some other code to reorganize the code.
* detect_tkinter() now returns False if the extension is missing.
* Add PyBuildExt.config_h_vars attribute
2019-03-01 16:43:28 +01:00
Victor Stinner 625dbf2567
bpo-36146: Refactor setup.py: Add PyBuildExt.srcdir (GH-12124)
* Add PyBuildExt.srcdir atribute in setup.py: the source directory is
  now always absolute.
* Add PyBuildExt.inc_dirs and PyBuildExt.lib_dirs attributes:
  replace 'inc_dirs' and 'lib_dirs' local variables of
  detect_modules().
* Replace "from distutils.errors import *"
  with "from distutils.errors import CCompilerError, DistutilsError"
  to be able to use static analyzers like pyflakes
* Reorder imports.
2019-03-01 15:59:39 +01:00
Victor Stinner 8058bdae3e
bpo-36146: Refactor setup.py: PyBuildExt.add() method (GH-12097)
* Add PyBuildExt.add() which adds the extension directly to
  self.extensions, rather than using a temporary 'exts' local
  variable in detect_modules() and then add 'exts' to self.extensions
* Convert 'missing' local variable from detect_modules()
  into PyBuildExt.missing attribute
* _detect_openssl(), _decimal_ext() and _detect_nis() now call
  directly self.add(), rather than returning an extension
  (or None if not found).
* Rename _decimal_ext() to _detect_decimal() for consistency with
  other methods.
2019-03-01 15:31:45 +01:00
Victor Stinner 96d81583be
bpo-36146: Fix inc_dirs in setup.py on macOS (GH-12098)
Fix setup.py on macOS: only add /usr/include/ffi to include
directories of _ctypes, not for all extensions.
2019-03-01 13:53:46 +01:00
Victor Stinner 4cbea518a0
bpo-36146: Refactor setup.py (GH-12093)
* Rename globals to upper case to better distinguish if a variable is
  global or local:

  * Rename cross_compiling to CROSS_COMPILING
  * Rename host_platform to HOST_PLATFORM
  * Rename disabled_module_list to DISABLED_MODULE_LIST

* Add MS_WINDOWS, CYGWIN and MACOS constants.
* Use booleans: replace "return 0" with "return False"
  and replace "return 1" with "return True"
2019-02-28 17:48:38 +01:00
pxinwr 32f5fdd7f4 bpo-31904: Add cross-build support for VxWorks RTOS (GH-11968) 2019-02-27 12:09:28 +01:00
Anthony Sottile 8377cd4fcd Clean up code which checked presence of os.{stat,lstat,chmod} (#11643) 2019-02-25 23:32:27 +01:00
Neil Schemenauer 5741c45acf
bpo-35903: Use autoconfig to probe for shm_open() and shm_unlink(). (#11765)
Use autoconfig to probe for shm_open() and shm_unlink().  Set SHM_NEEDS_LIBRT if we must
link with librt to get the shm_* functions.  Change setup.py to use the autoconfig defines.  These
changes should make it more likely that _multiprocessing/posixshmem.c gets built correctly on
different platforms.
2019-02-08 10:48:46 -08:00
Davin Potts e5ef45b8f5
bpo-35813: Added shared_memory submodule of multiprocessing. (#11664)
Added shared_memory submodule to multiprocessing in time for first alpha with cross-platform tests soon to follow.
2019-02-01 22:52:23 -06:00
stratakis cf10a750f4 bpo-35257: Avoid leaking LTO linker flags into distutils (GH-10900)
When compiling 3rd party C extensions, the linker flags used by the
compiler for the interpreter and the stdlib modules, will get
leaked into distutils. In order to avoid that, the PY_CORE_LDFLAGS
and PY_LDFLAGS_NODIST are introduced to keep those flags separated.
2018-12-19 18:19:01 +01:00
Victor Stinner 130893debf
bpo-35081: Internal headers require Py_BUILD_CORE (GH-10363)
* All internal header files now require Py_BUILD_CORE or
  Py_BUILD_CORE_BUILTIN to be defined.
* _json.c is now compiled with Py_BUILD_CORE_BUILTIN to access
  pycore_accu.h header.
* Add an example to Modules/Setup to show how to build _json
  as a built-in module; it requires non trivial compiler options.
2018-11-09 13:03:37 +01:00
Chih-Hsuan Yen 09b2bece78 bpo-29442: Replace optparse with argparse in setup.py (GH-139) 2018-07-11 17:48:43 +09:00
Brett Cannon a8c342465b
bpo-25427: Remove pyvenv (GH-5962) 2018-04-20 14:15:40 -07:00
Christian Heimes 5bb9692575
bpo-32647: Link ctypes extension with libdl. (#5550)
The ctypes module used to depend on indirect linking for dlopen. The shared
extension is now explicitly linked against libdl on platforms with dl.

Signed-off-by: Christian Heimes <christian@python.org>
2018-02-25 10:22:14 +01:00
Ivan Levkivskyi 03e3c340a0
bpo-31333: Re-implement ABCMeta in C (#5273)
This adds C versions of methods used by ABCMeta that
improve performance of various ABC operations.
2018-02-18 12:41:58 +00:00
Eric Snow 7f8bfc9b9a
bpo-32604: Expose the subinterpreters C-API in a "private" stdlib module. (gh-1748)
The module is primarily intended for internal use in the test suite.  Building the module under Windows will come in a follow-up PR.
2018-01-29 18:23:44 -07:00
Christian Heimes 61d478c71c
bpo-31399: Let OpenSSL verify hostname and IP address (#3462)
bpo-31399: Let OpenSSL verify hostname and IP

The ssl module now uses OpenSSL's X509_VERIFY_PARAM_set1_host() and
X509_VERIFY_PARAM_set1_ip() API to verify hostname and IP addresses.

* Remove match_hostname calls
* Check for libssl with set1_host, libssl must provide X509_VERIFY_PARAM_set1_host()
* Add documentation for OpenSSL 1.0.2 requirement
* Don't support OpenSSL special mode with a leading dot, e.g. ".example.org" matches "www.example.org". It's not standard conform.
* Add hostname_checks_common_name

Signed-off-by: Christian Heimes <christian@python.org>
2018-01-27 15:51:38 +01:00
Christian Heimes 29a7df7827
bpo-32521: nis libnsl (#5190)
The nismodule is now compatible with new libnsl and headers location

Signed-off-by: Christian Heimes <christian@python.org>
2018-01-26 23:28:46 +01:00
Yury Selivanov f23746a934
bpo-32436: Implement PEP 567 (#5027) 2018-01-22 19:11:18 -05:00
Victor Stinner 13ff24582c
bpo-32593: Drop FreeBSD 9 and older support (#5232)
Drop support of FreeBSD 9 and older.
2018-01-22 18:32:50 +01:00
Christian Heimes ff5be6e810
bpo-32598: Use autoconf to detect usable OpenSSL (#5242)
Add https://www.gnu.org/software/autoconf-archive/ax_check_openssl.html
to auto-detect compiler flags, linker flags and libraries to compile
OpenSSL extensions. The M4 macro uses pkg-config and falls back to
manual detection.

Add autoconf magic to detect usable X509_VERIFY_PARAM_set1_host()
and related functions.

Refactor setup.py to use new config vars to compile _ssl and _hashlib
modules.

Signed-off-by: Christian Heimes <christian@python.org>
2018-01-20 13:19:21 +01:00
Antoine Pitrou 94e1696d04
bpo-14976: Reentrant simple queue (#3346)
Add a queue.SimpleQueue class, an unbounded FIFO queue with a reentrant C implementation of put().
2018-01-16 00:27:16 +01:00
Christian Heimes f3031b8a7a
bpo-32521: nis libtirpc (#5137)
glibc has removed Sun RPC. Use replacement libtirpc headers and library in
nis module

Signed-off-by: Christian Heimes <christian@python.org>
2018-01-12 15:26:32 +01:00
Mike 53f7a7c281 bpo-32297: Few misspellings found in Python source code comments. (#4803)
* Fix multiple typos in code comments

* Add spacing in comments (test_logging.py, test_math.py)

* Fix spaces at the beginning of comments in test_logging.py
2017-12-14 13:04:53 +02:00
Stefan Krah bd4ed77f73
bpo-32233: Fix build with --with-system-libmpdec. (#4739) 2017-12-06 18:24:17 +01:00
Nick Coghlan 53efbf3977
bpo-11063: Handle uuid.h being in default include path (GH-4565)
find_file() returns an empty list if it finds the requested
header on the standard include path, so header existence
checks need to be explicitly against "is not None".
2017-11-26 13:04:46 +10:00
xdegaye 77f5139954
bpo-32059: setup.py now also searches the sysroot paths (GH-4452)
detect_modules() in setup.py now also searches the sysroot paths when
cross-compiling.
2017-11-25 17:25:30 +01:00
Benjamin Peterson 8acaa31eec
remove detect_math_libs (#4383)
Darwin may not require libm, but it doesn't hurt to link it and simplifies configuration logic.
2017-11-12 20:53:39 -08:00
Victor Stinner 8c663fd60e
Replace KB unit with KiB (#4293)
kB (*kilo* byte) unit means 1000 bytes, whereas KiB ("kibibyte")
means 1024 bytes. KB was misused: replace kB or KB with KiB when
appropriate.

Same change for MB and GB which become MiB and GiB.

Change the output of Tools/iobench/iobench.py.

Round also the size of the documentation from 5.5 MB to 5 MiB.
2017-11-08 14:44:44 -08:00
Michał Górny 1aa00ff383 fixes bpo-31834: Use optimized code for BLAKE2 only with SSSE3+ (#4066)
Rework the code choosing BLAKE2 code paths from using the optimized
variant on all x86_64 machines to using it when SSSE3 or better
supported instructions sets are available.

Firstly, this solves the problem of using pure SSE2 code path on x86_64
machines. As reported in the bug, this code is slower than the reference
code on all tested x86_64 machines. Furthermore, on Athlon64 that lacks
SSSE3, it is even 2.5 times slower than the reference code! Checking
for SSSE3 therefore ensures that the optimized implementation will only
be used when it has a chance of performing better.

Secondly, this makes it possible to use SSSE3+ optimizations on 32-bit
x86 systems. This allows for even 2 times speed gain on modern 32-bit
x86 systems (tested in a 32-bit chroot).
2017-10-23 23:54:19 -07:00
Benjamin Peterson edc05c5d88 remove comment about long-gone SGI modules (#3850) 2017-10-01 23:37:57 -07:00
Antoine Pitrou a106aec2ed bpo-11063, bpo-20519: avoid ctypes and improve import time for uuid (#3796)
bpo-11063, bpo-20519: avoid ctypes and improve import time for uuid.
2017-09-28 23:03:06 +02:00
Antoine Pitrou a6a4dc816d bpo-31370: Remove support for threads-less builds (#3385)
* Remove Setup.config
* Always define WITH_THREAD for compatibility.
2017-09-07 18:56:24 +02:00
Shlomi Fish 6d51b87612 bpo-30912: Don't check the content of ffi.h (GH-2687)
Various platforms have various methods of handling multiarch libffi which probably won't match the previously looked-for defines.  Now we just make sure that ffi.h is available.
2017-09-06 13:19:19 -07:00
Devin Jeanpierre c5bace2bf7 bpo-29505: Add fuzz tests for float(str), int(str), unicode(str) (#2878)
Add basic fuzz tests for a few common builtin functions.

This is an easy place to start, and these functions are probably safe.
We'll want to add more fuzz tests later.  Lets bootstrap using these.

While the fuzz tests are included in CPython and compiled / tested on a
very basic level inside CPython itself, the actual fuzzing happens as
part of oss-fuzz (https://github.com/google/oss-fuzz). The reason to
include the tests in CPython is to make sure that they're maintained
as part of the CPython project, especially when (as some eventually
will) they use internal implementation details in the test.

(This will be necessary sometimes because e.g. the fuzz test should
never enter Python's interpreter loop, whereas some APIs only expose
themselves publicly as Python functions.)

This particular set of changes is part of testing Python's builtins,
tracked internally at Google by b/37562550.

The _xxtestfuzz module that this change adds need not be shipped with binary distributions of Python.
2017-09-06 11:15:35 -07:00
Stefan Krah 9e1e6f528f bpo-30923: Silence fall-through warnings in libexpat build. (#3205) 2017-08-25 14:07:50 +02:00